Low-rank optimization with convex constraints

The problem of low-rank approximation with convex constraints, which often appears in data analysis, image compression and model order reduction, is considered. Given a data matrix, the objective is to find an approximation of desired lower rank that fulfills the convex constraints and minimizes the distance to the data matrix in the Frobenius-norm. The problem of matrix completion can be seen as a special case of this. Today, one of the most widely used techniques is to approximate this non-convex problem using convex nuclearnorm regularization. In many situations, this technique does not give solutions with desirable properties. We instead propose to use the optimal convex minorizer – the closed convex hull – of the Frobenius-norm and the rank constraint as a convex proxy. This optimal convex proxy can be combined with other convex constraints to form an optimal convex minorizer of the original non-convex problem. With this approach, we get easily verifiable conditions under which the solutions to the convex relaxation and the original non-convex problem coincide. Several numerical examples are provided for which that is the case. We also see that our proposed convex relaxation consistently performs better than the nuclear norm heuristic, especially in the matrix completion case. The expressibility and computational tractability is of great importance for a convex relaxation. We provide a closed-form expression for the proposed convex approximation, and show how to represent it as a semi-definite program. We also show how to compute the proximal operator of the convex approximation. This allows us to use scalable first-order methods to solve convex approximation problems of large size.